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ABSTRACT 

Focusing on how the definition and assessment of 
quality day care are culturally specific, this paper reviews the 
development and validity of the Child Care Facility Schedule (CCFS) , 
a procedure for assessing quality in child care settings in many 
countries. This project was initiated by the Division of Mental 
Health of the World Health Organization, and began with a pilot study 
undertaken in Greece, Nigeria, and the Philippines. This initial 
study resulted in an 80-item schedule covering 8 areas that define 
quality child care: physical environment; health and safety; 
nutrition and food services; administration; s taf f -f amily 
interaction; staff-children interaction; observable child behavior; 
and curriculum. Concurrent validity, criterion validity, and 
construct validity were examined by comparing CCFS scores in 12 day 
care centers with ratings based upon observation in the same centers. 
An additional study of 90 day care centers in Athens, Greece, further 
established CCFS validity. Results showed that the CCFS was reliable 
and valid. The use of a shorter 43~item version is suggested to 
render the measure more practical. (MM) 
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Introduction 

The main aim of today's presentation is to describe 
the development of an instrument for the assesssment of 
quality in day-care settings for use in different cul- 
tures, and to demonstrate its reliability and validity 
with a sample of day-care centres. To achieve this it was 
decided to examine several types of validity. 

The debate on whether day care is, compared to mater — 
nal care, beneficial or not for children has moved on to 
questions concerning the quality of day care provided 
(Phillips and Howes, 1987). Based on evidence stemming 
from the longitudinal research into day care, it has been 
argued that where the day care is meeting the needs of the 
child in certain areas better than they could be met at 
home the child will benefit from being in the day care 
setting (Hennessy and Melhuish, 1991). 

In other words optimal child development appears to be 
linked to high quality day care. Yet researchers, planners 
and clinicians as well, do not seem to have reached a con- 
sensus on defining and assessing quality care (Kontos and 
Stevens, 1985). Questions that have been raised relate to 
the hypotheses being addressed, the domain of child out- 
come being measured and the approach to measuring quality 
(Phillips and Howes, 1987; Zaslow, 1991). As summarized by 
Phillips and Howes (1987) research concerning the approach 
to measuring quality, primarily in the United States, has 
focused either on the global aspect of quality reflecting 
the overall climate of a program (e.g. the Day Care Envi- 



ronmental Inventory by Prescott, Kritchevsky and Jones, 
1972, and the Early Childhood Environment Rating Scale by 
Harris and Clifford, 1980) or on specific dimensions of 
child care. The latter encompasses structural aspects of 
child care, such as group composition and staff qualifica- 
tion ? dynamic aspects of child care concerned with chil- 
dren's daily experiences; and contextual aspects of child 
care, such as type of setting and staff stablity. 

Attempts at assessing quality day-care in non-Western 
cultures include an instrument which was developed in Mal- 
aysia tapping upon seven areas of care (Yusof, 1903) and a 
a set of questions and criteria which were developed for 
use in Latin America to help parents select day-care 
centres for their children (Etchegoyhen de Lorenzo and 
Newcomb, 19B4 ) . 

Defining quality is inherently a value-based issue 
which is faced so much by planners and policy makers as by 
researchers (Moss, 1991; Moss and Melhuish, 1991). The 
study reported here, emerged out of a concern regarding 
the extent to which cefinition and assessment of quality 
day care, being value-based, is culturally sr^cific. The 
Division of Mental Health of the World Health Organization 
(Geneva) initiated the development of a new instrument to 
be developed jointly by several cultures lying on differ — 
ent points on the continuum of social change. The aim was 
to provide a procedure for assessing quality in child-care 
settings for a broad range of countries. 
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The development of criteria for quality assurance of 
care, ensuring both the physical and psychosocial well- 
being, lies among the interests of WHO'?: Mental Health Pro- 
gramme* 

Preliminary research on the Child Care Facility Sched- 
ule (CCFS) was carried out in 1983 by a Collaborative 
Child Care Project working group. The pilot study was 
undertaken by three participant countries, namely Greece, 
Nigeria and the Philippines. 

A starting point for work on the Schedule was a care- 
ful review of existing criteria, particularly those 
developed by the U.S. National Association for the Educa- 
tion o-f Young Children for accrediting centres in the 
United States. These criteria were recast into simple 
declarative statements, reflecting conditions that could 
easily be observed upon a visit to a centre or else that 
called for information that could be secured through a 
brief interview (Caldwell, 1984). 

By consultation with colleagues from the three pilot 
centres a draft 80-item schedule covering the important 
areas of day-care operation was created. Emphasis was 
p 1 aced on the relevance to each country involved and on 
ease of scaring. Items were translated by a translator 
into each national language and then back-translated into 
English by another translator. The back translations were 
then checked for meaning and accuracy. 

This draft was used in a number of day- care centres 
representing different levels of quality ( high-modera te- 



low) in Athens (N=20), Manila (N=15) and Ibadan N=91 ) . 
The items chosen were found to be relevant in each setting 
and we^e easy to apply. Training of the observers in the 
use of the schedule was established. The length of the 
observation time was two hours approximately. 

The result of this initial undertaking was the cre- 
ation of an 80-item schedule covering eight important 
areas that define quality child-care: Physical Environ- 
ment, Health and Safety, Nutrition and Food Services, 
Administration , Staf f-Fami ly Interaction , St af f-Chi ldren 
Interaction, Observable Child Behaviour, and Curricu- 
lum. (TABLE 1) 

Subsequently, an inter-rater reliability study was 
taken up in both Athens and Ibadan, with a sample of 69 
day-care centres in total, to check the consistency of 
ratings by different observers on different occasions, 
while remaining blind to each other's ratings. The time 
interval between ratings ranged from six to ten days in 
the Greek study, and one to four weeks in the Nigerian 
study. Reliability ranged from .83 for the Health and 
Safety categury to .99 for the Nutrition and Food Service 
category, in the Greek study, while overall reliability 
was .94 in the case of Nigeria (Tsiantis et al., 1991). 
Validity 

The establishment of the validity of an instrument 
undoubtedly remains one of the very difficult problems in 
psy chometrics and there is no one satisfactory solution. 
Thus researchers need to assess the validity of an 



instrument in a number of ways. Concurrent validity, 
criterion validity and construct validity were examined in 
the present case. 
Method 

Details of the validity procedures 

Concurrent validity was examined by comparing the CCFS 
scores with ratings based upon observation. An experienced 
observer spent a full day in each of twelve day-care 
centres, at a different time than the interviewer who completed 
the CCFS. The observer examined the environment, the cli- 
mate and the interactions, the daily curriculum and the 
activities in general while she remained non-paricipatory 
She was not familiar to the content of the Schedule but 
she was instructed to focus her observation on specific 
areas. These areas were in Tact the eight areas of care 
defined by the CCFS. At the end of the second day the 
observer assessed each day-care centre on the above dimen- 
sions on a 4-point scale, similar to the CCFS one. 

Criterion validity was ascertained by comparing the 
CCFS scores with the scores resulting from another, simi- 
lar in content and scope, widely used scale, the Early 
Childhood Environment Rating Scale (ECERS) by Harms and 
Clifford (19BO). Thus quality of every single day-care 
centre was assessed by both scales. 

The construct validity was ascertained by a factor 
analytic method. The construct validity of a test presup- 
poses a process of three phases. The first phase in the 
present case, refers to the development of the hypotheses 
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and assumptions raised by the initial members of the WHO 
Collaborative Child Care Project working group regarding 
the criteria for high quality programs. The second phase 
of construct validity establishment refers to the gradual 
development of the Schedule and the creation of the eight 
areas which define quality of day-care. The third phase 
concerns the empirical testing of those. 
Sample 

A sample of 90 day-care centres in the Athens area 
were selected with random stratified sampling. The sample 
represents 15*/. of the total number of day-care centres in 
the area and is representative of every type of centre; 
state-operated (N=45) private ( N=*29, including the ones 
created by the trade-union of big public corporations), 
those belonging to pubLic welfare institutions (N=10) and 
those operated by the municipality of Athens <N=5). Four 
interviewers visited about 23 day-care centres each. They 
were previously rigorously trained and satisfactory 
reliability (r = .93) among the interviewers was secured. 
Resul ts 

In examining concurrent validity the Pearson product- 
moment correlation coefficient between the overall ratings 
of the observer and those of the CCFS was obtained. The 
correlation was r = ,81 (p < .01). In other words the 
observer and the interviewers who completed the Schedule 
assessed the same characteristics in a similar fashion. 

Moreover, the mean scores of the two assessments were 
compared for each area of concern, separately. A paired 
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t-test revealed that far the areas of "Physical Environ- 
ment" , "Administration" f "Staff -Chi Idren Interaction" , 
"Observable Child Beahaviour and "Curriculum" there were 
no differences between the observer's mean ratings and the 
CCFS scores. The t-values differed in three areas. These 
were "Health and Safety", "Nutrition and Food Services" 
and "Staff -Fami ly Interaction". These differences were 
attributed to the fact that assessment of those areas is 
based much less on direct observation and much more on the 
response to questions presented to the personnel by th£ 
interviewer . 

Criterion validity was tested by comparing the CCFS 
scores with the ECERS ones. The Pearson product-moment 
correlation coefficient between the two sets of scores 
which were f irst transformed into standard z -scores , was 
r =.80 ( p <.001). This correlation was considered very 
satisfactory. Correlation between a new instrument and a 
similar oarlier one is an evidence that the new one, mea- 
sures approximately the same general concept as the other 
( Anastasi , 1976) . 

The overall mean scores of the two instruments were 
compared with paired t-tests. The t-values were signifi- 
cantly different (ECERS: 2.66, CCF5:3.05, t =-14.07, 
p <.000). The higher CCFS score could possibly be attrib- 
uted to the fact that the ECERS focuses more than does the 
CCFS, on characteristics relating to the curriculum, to 
the sophistication of materials and to language/reasoning 
activities. In other words the ECERS seems to focus on 
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somewhat different dimensions of quality . Moreover, the 
CCFS is on a 4-point scale while the ECERS is on a 7-point 
scale, hence probably more difficult for raters to use. 

A principle components factor analysis with varimaK 
rotation was performed in order to test for construct val- 
idity. In an eight factor solution ? following the areas 
designated originally, the first factor appeared to be 
very strong providing thus more like a one factor solu- 
tion. The total variance explained is 577.; the first fac- 
tor contributing 30"/. and the second 7V. (Tables i and 2). 
The rest of the factors explain very small percentages 
each, and they include few significant variables which 
make no clear meaning. The first factor includes char — 
acteristics from all eight areas. In other words, it is 
the case of a one overwhelming goodness factor •( TABLES 2 
and 3) 

A further factor analysis was completed requesting 
only four factors, in order to compare the results with 
those obtained by the first analysis. The first factor 
was just as strong and the content was very similar to the 
one which emerged from the first analysis. 

Looking at the structure of the first factor the 
greater number of significantly loading items (eigen- 
value > .35), are characteristics alluding to the human 
element responsible for quality day-care. A large number 
of variables included in this factor refer to those 
aspects of personnel , curriculum and environment which 
encourage and promote a calm and pleasant atmosphere, ere- 
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ativity and decision making in children, group participa- 
tion and individuation at the same time. The structure of 
the second, much less important, factor refers more to the 
infrastructure of the day-care. In other words, quality 
of care is greatly attributed to the human factor. It is 
noted that other Greek and international attempts of 
quality day-care assessment, have revealed one general 
quality factor as well (Lambidi and Todoulou, in prepara- 
tion; McCartney, 1984; Phillips, McCartney and Scarr, 
1987) . 

Thus, while the constructs initially conceived as 
representing quality day-care are being reconfirmed, the 
eight distinct areas, were not clearly discriminated. 
Moreover the items which loaded significantly onto the two 
first factors are only 43, a lot fewer that the initial 
number of 80 items constituting the Schedule. The CCFS 
scores of the shorter 43-item version were once more cor — 
related with the ECERB scores. This correlation was found 
to be r ~ .76 (p <.001). This means that the Schedule may 
be shortened, becoming thus much easier to administer. 

A cluster analysis of the 90 day-care centres revealed 
three clusters; the third being a very small one compris- 
ing only five day-care centres. The overwhelming majority 
of centres is included in the first two clusters. Fifty 
nine centres are included in the first cluster and 20 in 
the second. Looking at the composition of each cluster it 
was revealed that the first cluster is comprised of centres 
where the quality of care is good while the other is com- 



prised of centres where the quality is much poorer. Thus 
again this dichotomy of goodness versus badness in tne 
qual ity provided , was apparent . 
Assessment of quality day-care 

The CCFS scores were used to compare the quality of 
care provided by the different types of day-care centre 
(state-operated, private, operated by the municipality of 
Athens, and by public welfare institutions). 

A one-way ANOVA revealed that private day-care centres 
had the highest scores ( F = 9.38, p < .0000, df 3). The 
Scheffe procedure denoted that the significantly different 
pair at the .05 level, was the state-operated and private 
day-care centres. This difference was in the expected 
direction judging from previous findings with a smaller 
r&spective sample of day-care centres (Tsiantis et al . , 
1988). However, it must be taken into consideration that 
the number of day-care centres operated by the municipal- 
ity and by public welfare institutions is proportionately 
very small sine? it was a stratified sampling. 

Looking at tn? previous data, the range of CCFS scores 
derived from observation of the private day-care centres 
was very wide; i.e. the higher and the lower CCFS scores 
were given to private centres (ibid). A rank order of the 
day-care centres in the present study, revealed that 87V. 
of the centres belonging to the first quadrant are state- 
operated while 68% of the centres belonging to the fourth 
quadrant are private ones. Interestingly, the first two 
highest scores were given to centres operated by the 



12 

trade-union of the two biggest national banks in 
Greece. 

The four types of centres were also compared along the 
eight areas of care provision. On the dimension of Physi- 
cal Environment, private day-care centres had the highest 
scores (F - 6.46, p <.0005); the pair significantly dif- 
ferent being the private and the state-operated centres. 

For the Health and Safety provided at the day-care, 
all types of centres differed among themselves; the high- 
est mean score belonging to those centres operated by pub- 
lic welfare institutions and the second high to private 
centres (F - 6.80, p <.0004). The Scheffe procedure 
showed that three pairs were significantly different; the 
private and state-operated, the private and the ones oper — 
ated by welfare institutions, and the state-operated and 
the ones operated by welfare institutions. No differences 
were noted for the Nutrition and Food Services category. 

The Administration was another area of great differ — 
ences. These differences were expected since the four 
types of centres vary inherently in this respect. Private 
centres had the highest score (x = 2.98), the second being 
the centres operated by the municipality ( x = 2.B9) 
(F ~ 26.03, p <.0000). The only two pairs which were not 
significantly different were the private and municipal 
ones, and the ones operated by welfare institutions and 
the municipality. 

In terms of Staff-Family Interaction, the differences 
noted were the ones between the state-operated and the 
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private ones, the latter having higher scores (F = 3*56, 
p<»01). Similar was the case -for the Children-Family 
Interaction (F * 5.29, p =.002). 

In the area of Observable Child Behaviour, the mean 
scores differed among the various types, the highest score 
alloted to the centres operated by the municipality 
(x = 3.57) and the second high to the private ones 
(x = 3.42). Yet the significantly different pair was the 
state-operated and the private centres. Finally, as far 
as the Curriculum is concerned private centres had the 
highest scores and the significant difference remained the 
one between private and state-operated centres. 
Discussion 

The results show that the WHO Child Care Facility 
Schedule is a reliable instrument and a valid one, in a 
variety of ways. Concurrent and criterion validity were 
satisfactory. The observation conducted by the observer 
who was unfamiliar to the content of the Schedule, yielded 
similar scores to the on^s provided by the CCFS. The cor — 
relation between the CCFS and the ECERS score was quite 
high, confirming that the new instrument is similar enough 
but has its own distinct character and is not a mere 
dupl icat ion • 

Testing for construct validity an overall quality 
factor was revealed showing once more that good things go 
together in child-care. The use of a shorter 43-item ver- 
sion of the Schedule is suggested, thus rendering it more 
practical. Looking at the first two factors, it seems that 
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the human element and its contribution to personal and 
collective differentiation, is of paramount importance 
while the technical infrastructure is secondary. At times 
when the emphasis is greatly directed towards material 
goods and technical perfection, the above results provoke 
our priorities. This certainly does not imply that appro- 
priate materials and space are not indispensary presuppo- 
sitions for the maximization of both the educators' and 
the children's potential. Furthermore the results of the 
cluster analysis confirmed the fact that centres belonging 
to the "high quality " cluster rank high in both psychoso- 
cial and physical aspects of care. 

In using the CCFS we were able to identify differences 
among the various types of day-care centres in the 
expected direction. The quality of care provided by the 
private day-care centres was shown to be higher in previ- 
ous and current work (Tsiantis et al., 1988; Lambidi and 
Todoulou, in preparation). Efforts of organizing care 
where the private initiative represented by differentiated 
either centre owners or personnel syndicates, seem to 
secure high quality care. 

In closing, I would like to suggest that a centre's 
personnel can also benefit from the use of the CCFS for 
self-evaluation, in terms of the provision of a structure 
for setting goals for improvement in their work . 
Already in Portugal the CCFS was used for such a purpose 
with 250 centres in the Lisbon area (Caldeira et al., 
1992) . 
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Finally, it should be stressed that issues of quality 
definition, assessment and attainment in the field of 
child-care, are not to remain only within the realm of the 
researcher's interests. There are key policy issues and 
are intricately connected to the challenges faced by plan- 
ners in the field of primary prevention. 
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Table i Areas covered by the CCF5 



1. Physical Environment 

2. Health and Safety 

3. Nutrition and Food Services 

4. Administration 

5. Staff-Family Interaction 

6. Staf f-Chi ldren Interaction 

7. Observable Child Behaviour 

8. Curriculum 



20 



20 



Table 2 First CCFS Rotated Factor 



Items Loadings 

3. Sounds are moderate and pleasant (PE) .56 

4 .Sufficient and appropriate materials (PE) .46 

6. Areas available where children can be somewhat .38 
alone occasionally (PE) 

14. Centre meets standards for protection of health .47 
and safety (HS) 

20. Staff attentive to each child's health and .57 

development (HS) 

28. Meal times used to promote good nutrition (NFS) .45 

51. Regular training opportunities for staff to .38 

improve skills in working with children (A) 

56. At least once a year a staff member individually .45 

discusses child's progress with parents (SFI) 

59. Staff encourage children to share experiences .70 

and feelings and are responsive to chi Idrens ' needs ( SCI ) 

60. Staff speak to children in friendly, positive .67 

and courteous manner (SCI) 

61. Staff respect chi Idrens ' cul tura 1 backgrounds ( SCI ) .71 

62. Staff encourage a degree of independence in child- .48 

ren compatible with their developmental matur i ty ( SCI ) 

63. Staff use positive reinforcement and encaura- .79 

gement ( SCI ) 

64. Staff respect child's right to choose not to .54 
participate in group activities occasional ly ( SCI ) 
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Table 2 continued 



65. Children appear to be comfortable, relaxed , happy .67 

and involved in their activities(OCB ) 

66. Children respect needs, feelings and property .65 
of others(OCB) 

67. Children respond appropriately to caregivers' .57 

reque5ts(0CB) 

69. Children are friendly to staff, to one another .57 
and to visi tors(OCB) 

70. Children do not become disorganized or unruly .62 
when changing from one activity to another(OCB) 

71. Realistic curriculum goals based on childrens' .66 
individual needs and interests(C) 

72. Daily schedule provides variety of activi ties(C) .60 

73. Developmental ly appropriate learning opportunities .76 
for children(C) 

74 . Act ivi ties foster positive sel f -concepts and .81 
social skills in children(C) 

75 . Teaching/ learning activities encourage language .77 
development and help children improve ability to think, 
reason, question and experiment (C) 

76. Teaching/ learning activities encourage creative .68 

expression (C ) 

77. Teaching/ 1 earning activities enhance physical ,6i 
development and skills(C) 
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Table 2 continued 



78. Teaching/ learning activities encourage good .65 
health habits(C) 

79. Time is allowed for children to choose their own -52 
activities and appropriate materials are provided(C) 



eigenvalue - 24.00, R - 30.0 

PE = Physical Environment, HS ~ Health and Safety, 
NFS 55 Nutrition and Food Services, A = Administration 
SF1 ~ Staff-Family Interaction, SCI = Staff -Chi ldren 
Interaction, DCB = Observable Child Behaviour, 
C = Curriculum 
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Table 3 Second CCFS Rotated Factor 



Items Loadings 

2. Indoor environment attractive and pleasant(PE) .50 

7. Outdoor area is safe(PE) .58 

8. Outdoor area provides space and opportunity for .38 

vigorous activities (PE) 

9. Separate area for sleeping is provided ( F-E ) .43 

10. Centre looks clean and well-cared for(PE) .70 

11. Toilets, soap and water accessible to children(PE) .58 

12. All rooms are wel l-lighted(PE ) .66 

13. All rooms are wel 1-ven ti lated (PE ) .65 
17 . Instructions for handling medical emergencies(HS) .37 

18. Centre assumes responsibility for safe arrival .48 
and departure of children(HS) 

19. Children carefully super vised (HS ) .45 
22. Staff follow everyday hygienic principles in .42 

looking after the children(HS) 

25. First-aid kit available on the premises(HS) .45 

29. Standards for serving food are complied with(NFS) -64 

30. Eating utensils properly washed and stored(NFS) .62 



eigenvalue = 5.5, R = .6.8 

PE = Physical Environment, HS =Health and Safety, 
NFS = Nutrition and Food Services 



